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Method and apparatus for integrating a wide keyboard in a small device 



The invention relates to a method and an apparatus for using a wide keyboard in- 
cluding a wide range of keys in small mobile devices. 

5 Mobile devices like mobile telephones, personal digital assistants and game con- 
soles have been planned small in size and light in weight to become handy and easy 
to keep with in any situation. One limiting feature in diminishing mobile devices is 
the usability of a keypad. The popularity of sms-messages (short message service) 
makes the easy and convenient use of a keypad even more important. Further there 

10 are many functions for example in wap (wireless application protocol) devices, 
which make it inevitable to have a wide range of keys available. 

There are devices in the market, which combine a standard mobile phone with per- 
sonal digital access (pda) features, like touch sensitive screens and larger keyboards. 
As important as the screen size and the graphical interface are, especially business 
15 users value also the capabilities of working with text, accessing menus and sending 
and receiving messages. Users do a lot of input manually with modem mobile 
phones. It is rather difficult to make a compromise between a small device and the 
keyboard big enough to be convenient to use. 

In new wap-enabling, smart devices keymats are made bigger to achieve better 
20 usability. For example in the Communicator 9210 device manufactured by Nokia 
Corporation, Espoo, Finland there is a revised keyboard, which includes slightly 
bigger concave keys than earlier models of the Conmiunicator series. "Conmfiunica- 
tor" is a registered trademark of Nokia Corporation. In so-called normal mobile 
phones there are many characters in one button, in order to reduce the actual num- 
25 ber of buttons. In personal digital assistant devices, for example in those manufac- 
tured by Palm Inc., Santa Clara, Califomia, USA, there have been introduced extra 
tools for writing. An extra tool can be for example a touch screen panel, which ac- 
cepts a written input on a valid area of a screen panel. Also it is publicly known to 
input the data by activating drawn pushbuttons or graphical images. Typically this 
30 is done by positioning a cursor and activating a selected operation with the aid of a 
mouse, trackball, pen, wand, stylus, etc. The use of these auxiliary means requires a 
stable enviroimnent and situation. 

In the publication EP 651543 there is introduced a personal communicator includ- 
ing a touch sensitive display. This mobile personal conmiunicator includes a casing 
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for a cellular phone, a modem and data processing system. On a touch sensitive dis- 
play it is possible to choose a next operation by touching a drawn button on a dis- 
play. Further by touching the screen it is possible to magnify graphical images, such 
as parts of a fax view and move the images around the display. In this publication 
5 the touch screen display comprises a pressure sensitive overlay on a liquid crystal 
display (LCD). 

The publication US 4 899 292 presents a computer coupled to a terminal haying a 
resistive touch screen. As the former one, also this pubUcation deals with images or 
image formatted data on a display, which are to be magnified. A tactile input is 
lO* done with the aid of a wand on the touch sensitive screen. The screen detects a volt- 
age signal, which is then converted into coordinates in known manner. 

The publication US 6 073 036 illustrates a mobile station with a touch sensitive in- 
put means and an automatic symbol magnification operation. A mobile station is 
provided with a touch sensitive input means, such as a touch sensitive display or a 

15 touchpad. The symbols or keys on a touch sensitive keypad can be implemented 
smaller in size, because the touched key is magnified so, that it can be distinguished 
before selecting it as an input. The magnification is implemented as if there would 
be a convex lens in the middle of the user and the screen. So the illusion seen by the 
user is non-linear magnification of the original symbol(s) touched. Subsequent de- 

20 tection of tihe tactile input and its location can be used for further magnification or 
for selecting the symbol. In this solution the whole used keypad, which can include 
for example numbers, alphanumeric characters or special characters, is depicted at a 
time. The size of one symbol in the screen is dependent on the resolution of the 
screen. After the magnification, all symbols are still depicted. The pointed, selected 

25 one(s) is (are) magnified by the expense of diminishing all the others. The non- 
linear, lens-like magnification of selected symbol(s) causes the decreasing of the 
other, non-selected symbols done in proportion to their distance from the selected 
one(s). 

The objective of the present invention is to implement a clear, well-designed and 
30 easily usable keyboard integrated in a small mobile device so, that keys are distin- 
guishable prior to selecting the input. 

The objectives of the present invention are achieved so, that a touch sensitive dis- 
play provides a user interface and is used as an input and output means. Further the 
keys are zoomed in response to activation of the touch sensitive display to be clear 
35 and determinable. 
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The present invention is characterized in what is said in the independent claims. The 
advantageous embodiments of the present invention are presented in the pendent 
subordinate claims. 

The present invention is built on a basic assumption that a touch screen is used both 
5 for input and output. Data is besides seen and achieved by the display, also inputted 
through it or at least in very close relationship with objects shown on the display. 
The diminishing of the device is established by picturing the keys on the touch sen- 
sitive display. The two-way, interactive display contains means for zooming display 
information so that a zooming function makes individual keys distinguishable. This 
10 makes even a small device usable and convenient. The advantage is, that the input 
means, such as a keyboard, can be zoomed on a display so that selected area of the 
keyboard is magnified for a predetermined range. Therefore it is possible to use 
originally small keys in usable size. This makes it possible to manufacture a small, 
simulated keyboard. At the same time it provides comfortable, convenient use of the 
15 known keyboard, which most users already are familiar with, such as normal type- 
writer or computer keyboard with keys arranged to be at their typical places. 

Typically there is a touch sensitive layer on the basic display, such as LCD. The 
touch sensitive display can be used for all manual input from choosing function 
keys to writing sms-messages and navigating in the net. The touch sensitive display 

20 expresses both the actual display for user to get the normal output and the keypad, 
keyboard or some functional keys for user to gain means for input. The input part of 
the display may vary in size, shape and content in different applications and in dif- 
ferent modes of function. For example if the device is used for phoning, the input 
needs mainly numerical keys and further the function key, which establishes the 

25 connection. If the user wants to send an sms-message, the most needed keys include 
letters known from the typewriting or computer keyboards. Some often used func- 
tions can be activated from dedicated keys. Some rare used, mode specific functions 
can be presented in the display and activated from the adjacent keys. 

In the following the present invention is introduced in more details with the accom- 
30 panying figures, in which 

Figure 1 illustrates an apparatus according to an advantageous embodi- 

ment of the present invention as a block diagram, 

Figure 2 illustrates a method according to an advantageous embodiment 

of the present invention as a flow diagram, and 
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Figures 3a - 3c illustrate a method according to an advantageous embodiment of 
the present invention. 

, In the following description of the present invention a central part is a touch sensi- 
tive display. It is obvious that it is not essential to the present invention, which type 
5 of touch sensitive panel is used. Further the means for touching the panel to indicate 
a tactile input can vary from a finger to any auxiliary input means. The touch 
screens are sensitive to touch and allow user to interact with the device by touching 
items on the screen. A touch screen includes a touch screen panel, a controller and 
driver software. A touch screen panel is a transparent or treuislucent panel attached 
10 externally on the original display. The touch screen panel registers touch events and 
passes these signals to the controller. The controller then processes the signals and 
sends the data to the processor. Software drivers translate touch events for example 
into mouse- or key pressing events. 

There are three types of touch screen technology: resistive, surface wave and ca- 
15 pacitive. A resistive touch screen panel is coated with a thin metallic electrically 
conductive and resistive layer. In a resistive touch screen a change in the electrical 
current is registered as a touch event and the coordinates impressing its situation are 
sent to the controller for processing. Resistive touch screen panels are generally 
more affordable than other type of touch sensitive panels, but they offer only 75% 
20 clarity. Further the metallic surface layer is vulnerable and can be damaged by sharp 
objects. Resistive touch screen panels are not affected by outside elements such as 
dust or water. 

Surface wave technology uses ultrasonic waves that pass over the touch screen 
panel. When the panel is touched, a portion of the wave is absorbed. This change in 
25 the ultrasonic waves registers the position of the touch event and sends this infor- 
mation to the controller for processing. Surface wave touch screen panels are the 
most advanced of the three types, but they can be damaged by outside elements. 
With this kind of touch screen panels it is important to differ real touches from 
background noise, etc. 

30 A capacitive touch screen panel is coated with a material that stores electrical 
charges. When the panel is touched, a small amount of charge is drawn to the point 
of contact. Circuits located at each comer of the panel measure the charge and send 
the information to the controller for processing. Capacitive touch screen panels 
must be touched with a finger unlike resistive and surface wave panels that can use 
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also other auxiliary means. Capacitive touch screens are not affected by outside 
elements and have high clarity. 

The invention works well with all kinds of known touch screen types. It should be 
noted that independence of touch screen type means also that the invention can cer- 

5 tainly take advantage of even future touch screen types that are not yet known at the 
priority date of this patent application. Especially if a certain touch screen type 
would inherently include the feature of producing an indication about the relative 
strength, i.e. pressing force of a touch, such an indication could most advanta- 
geously be used for the purposes of discrimination between touch types, which is 

10 described in more detail later. 

Figure 1 illustrates a block diagram of an apparatus used in an advantageous em- 
bodiment of the present invention. The block diagram shows only those display- 
and input-related functional blocks, which are important in implementing the pre- 
sent invention. It is obvious that the mobile device may include much more func- 
15 tional blocks, such as transceiver and receiver branches and memory units. 

There is a display 105, which is typically a liquid crystal display (LCD), in figure 1. 
Overlaid on the display 105 there is a touch screen panel 100, which implements the 
physical interface for touch conunands. Typically there is a backlight, on it a dis- 
play and on it a touch sensitive overlay. This combination acts both as input and 
20 output device. There may also be some chosen dedicated or soft keys 103 as input 
means, but these are of little importance to the present invention. 

A touch detector 101 winnows out the actual touches from other interfering factors. 
It depends on a type of a touch screen, how the actual touches are separated from 
fault detections. If the touch screen panel is a resistive one, the panel is not effected 

25 by outside elements, such as dust and water, and the activation of the input point on 
the panel can be done by finger or by some auxiliary instrument. The ultrasonic 
wave touch screen panels can be damaged by outside elements and are also sensi- 
tive for example to background noise. The touch screen panels employing capaci- 
tive technology are not affected by outside elements. The capacitive screens must be 

30 touched with a finger. 

A processor 107 receives conunands from a discrinninator 102 (and a keypress de- 
tector 104, if dedicated or soft keys exist). A discriminator 102 discriminates be- 
tween different touch conmiands in a predetermined way. It identifies for example 
that a quick touch acts for command ^select' and a long stable touch acts for 
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*zoom', moving touch means *pan\ etc. The different ways of input are identified 
and used for differentiating their functions. Advantageously the input is to be per- 
formed in a way users akeady are familiar with from a former use of keyboards. 
The discriminating of different inputs can be implemented for example so that a 
5 discriminator 102 is integrated in processor 107, which calculates times limits of 
touches or compares two detected, sequential locations of touches. 

A keypress detector 104 detects key presses of the dedicated or soft keys 103 and 
' conveys the detected information to a processor 107. A processor 107 initiates and 
• • executes various functions as conmianded and it includes needed timers and other 
10 accomplishing units. The processor 107 issues display-relating commands to a dis- 
play driver 106, which then converts display conunands from processor 107 into 
driving signals for a display element 105. The display 105 implements the final 
conversion from electrical to visual form. 

Figure 2 shows a method according to an advantageous embodiment of the present 
15 invention. In this embodiment the zooming is done once and it is done always be- 
fore the input is selected. This is implemented for example so, that there is some 
predetermined time limit between steps, so that a constant tactile input first zooms 
and then activates a detected key. It is also common to select the input by releasing 
a touch. The checking of time limits is not shown in figure 2. 

20 At first the display unit or the screen is divided into two portions in block 200 in 
some application specific way. The divided input and output areas are adjustable 
according to used application. This may be a user-adjustable feature so that the dis- 
play size remains constant and the two screens may vary in size in relation to each 
other according to users commands. This kind of function is known from computer 

25 displays. According the present invention the area can also be adjusted application 
specific so, that a keypad-part or so called input part of the display allocates as 
small display area as possible leaving maximum area of the display for the output 
part. This is advantageous when a user wants to use the display mainly for perform- 
ing data, for example when reading sms-messages (short message service) or 

30 browsing data fetched from the Intemet. On the other hand the user may maximize 
the keypad- or input area by minimizing the output area totally invisible, when only 
effective use of the input area is needed. In application specific adjusting the area 
needed for input part mainly depends of the number of symbols or keys on a spe- 
cific keypad, which can be for example numeric, alphanumeric or include only 

35 characters, function keys or all the mentioned. The actual number of keys that can 
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be displayed at a time depends also on the resolution of the display. The individual 
symbols need to be visually distinguishable, before the user can select the input. 

Next in block 201 it is checked, whether the input area of the display is touched. 
The touch sensitive panel can inform the processor of an input or the processor can 

5 detect inputs for example periodically. It also depends on the type of the touch 
screen, how the detection of the touches is done in detail. Firstly a touch screen 
panel can be resistive, so it is coated with thin, metallic, electrically conductive and 
resistive layer. In a resistive touch screen the touch is sensed as a charge in electric 
current. Secondly a touch screen panel can be implemented with ultrasonic waves 

10 passing over the touch screen panel. When the panel is touched, portion of the wave 
is absorbed. So the change in an ultrasonic wave registers'" a position of a touch 
event. The third possible touch screen panel type is capacitive, which is coated with 
a material that stores electrical charges. In this solution the circuits are located in 
each corner of the panel. When touched, a small amount of a charge is drawn to the 

15 point of the contact. The type of the touch screen panel also affects to the means 
used for activating symbols on a display. Typically it is advantageous to use a fin- 
ger, because it is easy to use. Additionally the device can contain some pointing 
means, such as a pen, for a fine adjustment. 

If no touch or input event is detected in block 201, the detection is searched again or 
20 waited for. The block 201 also includes the discrimination of different functions. In 
this embodiment the discriminating operation depends on a current mode of a de- 
vice. If the touch is detected in block 201, the current mode of the display is exam- 
ined. If the display view is not zoomed in block 202, the zooming is done according 
to a block 203. The zooming is implemented so, that the activated, touched portion 
25 of the displayed keyboard is magnified and the rest of the keyboard is not seen at 
all. The magnified area is the area touched by a user and a certain area surrounding 
it. In this magnified area the keys and symbols are distinguishable and a user can 
easily determine and select a key or a symbol. If the display view is already zoomed 
in block 202, the operation continues according to the block 204. 

30 In this embodiment the activated symbol is highUghted in block 205 according to 
the second detected touch in block 204. The highUghted touch point can be changed 
by sUding touch means on the display. After the zooming, the input symbol can be 
selected for example by releasing the touching point. When the touch point is re- 
leased, the highlighted symbol is displayed or if the highlighted key was a function 

35 key, the function is executed. After this the displayed view can either stay in a 
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zoomed mode or return back to a small, all viewing mode. Correspondingly the op- 
eration continues from the step 204 or from the step 201. 

The above is only one embodiment of the present invention and many modifications 
may be made in the scope of the invention. According to one embodiment of the 

5 present invention after the touch is detected in the step 201, the discriminator (102 
in fig.l) aparts a zooming event from for example a sizing of windows, such as ad- 
justing the size of an input or an output portion of the touch sensitive display. Typi- 
cally window sizing and panning events are indicated by sliding a tactile input mean 
on the display. The zooming can then he identified from the stable, non-moving 

10 touch or frorn round moving touch. Basically this can be implemented by detecting 
two sequential positions of a touch and checking, if there is a difference between the 
places. According to above described embodiment accompanied by the figure 2, the 
zooming takes always place before the selection of the input. In this embodiment 
the mode of a display is checked: if zooming is not done, it is implemented in step 

15 203, after the zooming is done, the detected touch indicates the selected input. It 
should be noted that the zooming can be done more than once. If the first view is for 
example the whole typewriting keyboard, for example for editing an sms-message, 
the zooming can be advantageously done two or three times in order to get a view 
of visually distinguishable, recognisable keys. On the other hand the input of num- 

20 bers from the numeric keypad can be done faster, if the zooming is done only once. 
According to one advantageous embodiment a user has an opportunity to choose the 
input key or symbol without any zooming or magnification. A user can use some 
auxiliary means, such as a pen or stylus, to point out a small key. The magnification 
may be implemented in sequential steps or it can be done stepless until a continuous 

25 touch sustains. The stepless magnification may be implemented so, that there actu- 
ally are very small magnification steps and those are produced and presented se- 
quentially fast enough for user to get an impression of continuous presentation. 

Further one affecting feature is the dimension of the magnification. The dimension 
or magnification rate is typically predetermined and it may be a keypad specific fea- 

30 ture, specified by an application or a mode, or the dimension can be determinable 
by a user. Display with numbers of small keys and details must be magnified so 
much, that the using is comfortable. Also it must be kept in mind that if the magni- 
fication is multiplex or the original keypad has less and larger keys, a multiple 
magnification may cause only one or few keys to be visible on the screen. This is 

35 not usually convenient for the user, because the magnified area can be miss- 
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selected, after which the correction is easier and faster to do, if also the adjacent 
keys are visible. 

Typically the present invention is implemented so, that a portions of the original 
keyboard can be viewed in magnified mode just by sliding a finger or other touch 

5 means to a side edge of the viewed keyboard portion. Touching to the side of the 
viewed keypad makes the view to scroll. This is most advantageous for users, who 
are familiar with the original keypad, so they will know what keys vvill come up 
next, beside the viewed ones. The users unfamiliar with used keypads can eitter 
scroll around on a keypad and find the symbol they are looking for, or alternatively 

10 go back to a non-zoomed mode and start the zooming again on a selected area. 

* * * ■ 

The touched symbol or key is highlighted in a block 205 as a visual sign of a se- 
lected key to a user. In a block 206 the actual selection of a key, such as a letter or a 
number, or activation of a function or a soft key can be indicated in few ways. The 
typical way is to release tfie touch, when the wanted key is highUghted. Another 
15 possible way to select and activate the wanted key is to keep the touch fixed in one 
place on a selected key a predetermined time, for example few seconds. Also ac- 
cording to an advantageous embodiment if the sensed pressure on the screen ex- 
ceeds some limit value, the key is activated. It is obvious to a person skilled in the 
art, that there are many ways to implement the present invention. 

20 Figures 3a - 3c show one method according an advantageous embodiment of the 
present invention. The mobile device is in an sms-message writing mode and the 
input 302abc and output 301 portions are advantageously adjusted by the mode. The 
display 300 in figure 3a can be a typical phone device or for example a small wrist- 
mountable device. In the display 300 there is an output portion 301, which has a 

25 cursor 304 showing the current place for input. Below the output portion 301 there 
is an input or a keyboard portion 302a. This input portion 302a is a simulated key- 
board viewed on a display 300. 

Buttons or keys 303 are soft or dedicated keys. Typically dedicated keys are used to 
implement some often used functions, such as call or cancel. There can also be soft 
30 keys for different, changing functions. The functions will change according to the 
view or mode of the display so, that the alternative corresponding to a specific but- 
ton is represented adjacent to it on a display view. This makes the use of changing, 
mode specific alternatives easy and fast. 
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Now user wants to produce an sms-message, in particular user wants to input a let- 
ter D. The viewed keyboard is arranged in thie same order as an original typewriter 
keyboard. Many users are familiar with these typical keyboards from computers and 
laptops, so a typical user knows in some accuracy where the wanted letter 'D' is 
5 situated in the keyboard. The user points roughly the area that the wanted letter is 
situated. The touch point of a user in figure 3a is situated on the letter 'D\ 

In figure 3b the detected touch point of the keyboard is zoomed as shown in key- 
board portion 302b. The selection can advantageously be done just by putting a fin- 
ger, or some pointing mean, on a touch screen. User can set up the speed of zocnn- 

10 ing, which advantageously is set from few milliseconds to few seconds. While 
. zooming the keyboard, a user can define his search by moving a pointer on the 
touch screen panel. A chosen key is for example highlighted to give the user a vis- 
ual feedback of a current, active selection. A key can be selected to be the input by 
releasing the touch off the touch screen, when the specific key is selected. Other- 

15 wise the pointer is positioned again roughly on the area to be zoomed, in this em- 
bodiment the area surrounding the letter *D\ 

Figure 3c illustrates the touch screen 302c after the keyboard is magnified or 
zoomed for a second time. The keys are now advantageously at the minimum the 
size of 7 X 7 millimeters, which corresponds the average size of a finger tip. There 

20 are also edges 305 rounding the keyboard. Because only few keys are seen, the user 
can scroll the keyboard by moving a pointer to the selected side on the edge. Now 
the letter *D' is selected by lifting the pointer off from it, when it is highlighted. The 
letter *D' is then inputted to the display portion 301 above. After the input, the 
magnified, zoomed keyboard 302c can be used in magnified mode or retumed to 

25 original mode, in which the whole keyboard can be seen, as illustrated in figure 3a. 



